Papers with long-context comprehension
Marathon: A Race Through the Realm of Long Context with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing long-context benchmarks do not accurately evaluate large language models’ comprehension and reasoning abilities in extended texts. |
| Approach: | They propose a new evaluation benchmark that adopts a multiple-choice question format and uses a multi-choke question format to assess the comprehension and reasoning skills of large language models. |
| Outcome: | The proposed benchmark provides a rapid, precise, and unbiased appraisal of the long-context comprehension skills of large language models. |
LongLeader: A Comprehensive Leaderboard for Large Language Models in Long-context Scenarios (2025.naacl-long)
Copied to clipboard
Pei Chen, Hongye Jin, Cheng-Che Lee, Rulin Shao, Jingfeng Yang, Mingyu Zhao, Zhaoyu Zhang, Qin Lu, Kaiwen Men, Ning Xie, Huasheng Li, Bing Yin, Han Li, Lingyun Wang
| Challenge: | LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases . |
| Approach: | They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website . |
| Outcome: | The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks. |
Joint Enhancement of Relational Reasoning for Long-Context LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | JERR is a graph-based reasoning framework for large language models . it enables LLMs to handle extended contexts with improved reliability and transparency . |
| Approach: | They propose a graph-based reasoning framework that integrates synopsis extraction, graph construction, and relational reasoning. |
| Outcome: | The proposed framework outperforms baselines on ROUGE and F1 metrics and achieves the highest scores on the LLM-Rater evaluation. |
R-CHAR: A Metacognition-Driven Framework for Role-Playing in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing role-playing structures lack cognitive consistency in complex scenarios . Existing models excel in math and coding tasks but lack coherent reasoning . |
| Approach: | They propose a metacognition-driven framework that enhances role-playing performance . experimental results show performance improvements across varying scenario complexities . |
| Outcome: | The proposed framework outperforms existing models in social intelligence tasks and shows strength in long-context comprehension and group-level social interactions. |
Long Chain-of-Thought Fine-tuning via Understanding-to-Reasoning Transition (2025.emnlp-main)
Copied to clipboard
Chenxin An, Zhihui Xie, Xiaonan Li, Ming Zhong, Shansan Gong, Lei Li, Jun Zhang, Jingjing Xu, Lingpeng Kong
| Challenge: | Existing research on long-context scaling in language models has focused on managing lengthy input prompts instead of producing long outputs. |
| Approach: | They propose a sequence-level curriculum learning framework that shifts a model’s focus from interpreting long chain-of-thoughts to generating them. |
| Outcome: | Experiments on rigorous reasoning benchmarks, including AIME24 and GPQA Diamond, show that the proposed approach surpasses standard fine-tuning by over 10% while maintaining robust performance on understanding tasks. |